Stackelberg游戏模型，领导者致力于制定策略，而追随者最能做出响应，它发现了广泛的应用程序，特别是针对安全问题。在安全环境中，目标是为了保护某些资产，使领导者计算一个最佳策略。在许多这些应用程序中，追随者实用程序模型的参数尚不确定。分布式优化优化通过允许在可能的模型参数上进行分配来解决此问题，而该分布来自一组可能的分布。目的是最大程度地提高预期的效用，相对于最坏情况下的分布。我们启动了分配稳定模型的研究，以计算最佳策略。我们考虑了对追随者公用事业模型的不确定性的正常形式游戏的情况。我们的主要理论结果是表明，在各种不确定性模型中，始终存在分布稳定的stackelberg平衡。对于一组有限的追随者实用程序函数，我们提出了两种算法，用于计算使用数学程序的分布强烈的Stackelberg平衡（DRSSE）。接下来，在一般情况下，存在无限数量的可能的追随者实用程序功能，并且不确定性在有限支撑的名义分布周围由Wasserstein Ball表示，我们给出了一个增量的基于混合组合编程的算法来计算最佳的算法分配稳定的策略。实验证实了我们在经典的Stackelberg游戏中算法的障碍，这表明我们的进近范围扩展到中型游戏。

我们研究随机的在线资源分配：决策者需要分配有限的资源来为随机生成的顺序派遣请求，以最大程度地提高奖励。通过练习，我们考虑了一个数据驱动的设置，在该设置中，请求独立于决策者未知的分布。过去已经对在线资源分配及其特殊情况进行了广泛的研究，但是这些先前的结果至关重要和普遍地依赖于一个实际上不可能的假设：请求总数（地平线）是决策者事先知道的。在许多应用程序（例如收入管理和在线广告）中，由于需求或用户流量强度的波动，请求的数量可能差异很大。在这项工作中，我们开发了在线算法，这些算法对地平线不确定性是可靠的。与已知的马环境形成鲜明对比的是，我们表明没有算法可以达到与视野不确定性无关的恒定渐近竞争比率。然后，我们引入了一种新型算法，该算法将双镜下降与精心选择的目标消耗序列结合在一起，并证明其达到了有限的竞争比率。从地平线不确定性增长时，我们的竞争比达到了最佳生长速率，我们的算法几乎是最佳的。

最近的一项工作已经建立了未耦合的学习动力学，以至于当所有玩家在游戏中使用所有玩家时，每个玩家的\ emph {sorex} $ t $ recretitions在$ t $中增长了polygarithmarithm，这是$ t $的指数改进，比指数级的改进，比传统的保证在无缩写框架。但是，到目前为止，这些结果仅限于具有结构化策略空间的某些类别的游戏，例如正常形式和广泛形式的游戏。关于$ o（\ text {polylog} t）$遗憾界限是否可以为一般凸和紧凑型策略集获得的问题 - 这在经济学和多种系统中的许多基本模型中都发生 - 同时保留有效的策略更新是一种重要的问题。在本文中，我们通过建立$ o（\ log t）$ player后悔的第一个未耦合学习算法来回答这一点凸和紧凑的策略集。我们的学习动力基于对适当的\ emph {升起}空间的乐观跟随领导者的实例化，使用\ emph {self-condcordant正规器}，这是特殊的，这不是可行区域的障碍。此外，我们的学习动力是可以有效地实现的，如果可以访问登录策略的近端甲骨文，从而导致$ o（\ log \ log \ log t）$ ter-ter-ter-tir-tir-tir-tir-tir-tir-tir-tir-tir-tir-tir-tir-tirceptimity;当仅假设仅对\ emph {Linear}优化Oracle访问时，我们还会给出扩展。最后，我们调整动力学以保证对抗性制度中的$ O（\ sqrt {t}）$遗憾。即使在适用先前结果的特殊情况下，我们的算法也会改善最先进的遗憾界限，无论是依赖迭代次数还是对策略集的维度的依赖。

专为单药加固学习（RL）设计的算法通常无法在两人零和零和游戏中收敛到平衡。相反，在2P0S游戏中近似NASH和量子响应平衡（QRE）的游戏理论算法通常对RL竞争，并且很难扩展。结果，这两种情况的算法通常是分别开发和评估的。在这项工作中，我们表明，单个算法是一种近端正则化的镜像下降的简单扩展，我们称之为磁性镜下降（MMD） - 尽管它们的基本差异都可以在两种情况下产生强大的结果。从理论的角度来看，我们证明了MMD在广泛的游戏中线性收敛到QRE-这是第一阶求解器首次证明线性收敛。此外，我们通过自我播放作为表格NASH均衡求解器应用，我们从经验上表明，MMD在正常形式和广泛的形式游戏中都具有全反馈（这是标准RL算法首次完成），在正常形式和广泛的形式游戏中产生竞争性竞争因此）以及MMD在黑盒反馈设置中经验收敛。此外，对于单人Deep RL，在一小部分Atari和Mujoco游戏中，我们表明MMD可以与PPO的结果竞争。最后，对于多代理Deep RL，我们显示MMD可以在3x3突然的黑暗中胜过NFSP。

Robotic teleoperation is a key technology for a wide variety of applications. It allows sending robots instead of humans in remote, possibly dangerous locations while still using the human brain with its enormous knowledge and creativity, especially for solving unexpected problems. A main challenge in teleoperation consists of providing enough feedback to the human operator for situation awareness and thus create full immersion, as well as offering the operator suitable control interfaces to achieve efficient and robust task fulfillment. We present a bimanual telemanipulation system consisting of an anthropomorphic avatar robot and an operator station providing force and haptic feedback to the human operator. The avatar arms are controlled in Cartesian space with a direct mapping of the operator movements. The measured forces and torques on the avatar side are haptically displayed to the operator. We developed a predictive avatar model for limit avoidance which runs on the operator side, ensuring low latency. The system was successfully evaluated during the ANA Avatar XPRIZE competition semifinals. In addition, we performed in lab experiments and carried out a small user study with mostly untrained operators.

The purpose of this work was to tackle practical issues which arise when using a tendon-driven robotic manipulator with a long, passive, flexible proximal section in medical applications. A separable robot which overcomes difficulties in actuation and sterilization is introduced, in which the body containing the electronics is reusable and the remainder is disposable. A control input which resolves the redundancy in the kinematics and a physical interpretation of this redundancy are provided. The effect of a static change in the proximal section angle on bending angle error was explored under four testing conditions for a sinusoidal input. Bending angle error increased for increasing proximal section angle for all testing conditions with an average error reduction of 41.48% for retension, 4.28% for hysteresis, and 52.35% for re-tension + hysteresis compensation relative to the baseline case. Two major sources of error in tracking the bending angle were identified: time delay from hysteresis and DC offset from the proximal section angle. Examination of these error sources revealed that the simple hysteresis compensation was most effective for removing time delay and re-tension compensation for removing DC offset, which was the primary source of increasing error. The re-tension compensation was also tested for dynamic changes in the proximal section and reduced error in the final configuration of the tip by 89.14% relative to the baseline case.

Learning enabled autonomous systems provide increased capabilities compared to traditional systems. However, the complexity of and probabilistic nature in the underlying methods enabling such capabilities present challenges for current systems engineering processes for assurance, and test, evaluation, verification, and validation (TEVV). This paper provides a preliminary attempt to map recently developed technical approaches in the assurance and TEVV of learning enabled autonomous systems (LEAS) literature to a traditional systems engineering v-model. This mapping categorizes such techniques into three main approaches: development, acquisition, and sustainment. We review the latest techniques to develop safe, reliable, and resilient learning enabled autonomous systems, without recommending radical and impractical changes to existing systems engineering processes. By performing this mapping, we seek to assist acquisition professionals by (i) informing comprehensive test and evaluation planning, and (ii) objectively communicating risk to leaders.

In inverse reinforcement learning (IRL), a learning agent infers a reward function encoding the underlying task using demonstrations from experts. However, many existing IRL techniques make the often unrealistic assumption that the agent has access to full information about the environment. We remove this assumption by developing an algorithm for IRL in partially observable Markov decision processes (POMDPs). We address two limitations of existing IRL techniques. First, they require an excessive amount of data due to the information asymmetry between the expert and the learner. Second, most of these IRL techniques require solving the computationally intractable forward problem -- computing an optimal policy given a reward function -- in POMDPs. The developed algorithm reduces the information asymmetry while increasing the data efficiency by incorporating task specifications expressed in temporal logic into IRL. Such specifications may be interpreted as side information available to the learner a priori in addition to the demonstrations. Further, the algorithm avoids a common source of algorithmic complexity by building on causal entropy as the measure of the likelihood of the demonstrations as opposed to entropy. Nevertheless, the resulting problem is nonconvex due to the so-called forward problem. We solve the intrinsic nonconvexity of the forward problem in a scalable manner through a sequential linear programming scheme that guarantees to converge to a locally optimal policy. In a series of examples, including experiments in a high-fidelity Unity simulator, we demonstrate that even with a limited amount of data and POMDPs with tens of thousands of states, our algorithm learns reward functions and policies that satisfy the task while inducing similar behavior to the expert by leveraging the provided side information.

Speech-driven 3D facial animation has been widely explored, with applications in gaming, character animation, virtual reality, and telepresence systems. State-of-the-art methods deform the face topology of the target actor to sync the input audio without considering the identity-specific speaking style and facial idiosyncrasies of the target actor, thus, resulting in unrealistic and inaccurate lip movements. To address this, we present Imitator, a speech-driven facial expression synthesis method, which learns identity-specific details from a short input video and produces novel facial expressions matching the identity-specific speaking style and facial idiosyncrasies of the target actor. Specifically, we train a style-agnostic transformer on a large facial expression dataset which we use as a prior for audio-driven facial expressions. Based on this prior, we optimize for identity-specific speaking style based on a short reference video. To train the prior, we introduce a novel loss function based on detected bilabial consonants to ensure plausible lip closures and consequently improve the realism of the generated expressions. Through detailed experiments and a user study, we show that our approach produces temporally coherent facial expressions from input audio while preserving the speaking style of the target actors.

We study the problem of graph clustering under a broad class of objectives in which the quality of a cluster is defined based on the ratio between the number of edges in the cluster, and the total weight of vertices in the cluster. We show that our definition is closely related to popular clustering measures, namely normalized associations, which is a dual of the normalized cut objective, and normalized modularity. We give a linear time constant-approximate algorithm for our objective, which implies the first constant-factor approximation algorithms for normalized modularity and normalized associations.